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A Qualitative and Quantitative Comparison of the Impact of Portfolio Assessment 
Procedures Versus Traditional Assessment in a College Physics Course 

Abstract 

Popular belief in alternative assessment procedures suggests that the use of student portfolios can 
help learners successfully organize and integrate newly acquired scientific knowledge. This two- 
group comparison study documents the use of student created portfolios in an algebra-based, 
college-level, introductory physics course. Sixteen students were assessed primarily using a 
portfolio-style assessment procedure. Nineteen students were assessed primarily using traditional, 
objective examinations. 

Both groups were given the same cumulative, multiple-choice final examination. All 
students completed a pre/post self-report survey of achievement in physics. There were no 
significant differences in learner achievement between the two groups on the final examination or 
on the self-report of achievement given before and after instruction. Analysis of two focus group 
discussions did, however, suggest that students assessed by portfolios feel less anxious about 
learning physics, devote considerable time to reading and studying outside of class, internalize and 
personalize the content material, and enjoy the learning experience. The results of this study 
suggest that portfolio-style assessment procedures support student achievement at least at the 
same level as traditional assessment procedures and appear to have additional benefits. 



Introduction 

Many educators have been introducing portfolio and performance assessments in a variety of classroom 
learning enviromnents in response to the need for more authentic and equitable science assessment strategies 
(Slater & Astwood, 1995; Slater, 1994; Slater & Ryan, 1993; Ryan & Kuhs, 1993; Arter & Spandel, 1992; Collins, 
1992; O'Neil, 1992). There are, however, only a limited number of studies that have looked at the affect portfolio 
assessment has on traditional examination performance (Herman and Winters, 1994). Although there has been a 
great deal of educational research attention dedicated to curriculum and instruction issues, research on assessment 
approaches are clearly needed in science education. Almost no comparison research exists in the realm of 
alternative science education assessment. Recommendations for the use of portfolio assessment may seem faddish 
in the absence of research evidence demonstrating that such assessment actually supports and encourages students' 
learning. 

In practice, alternative assessments have always been an integral part of higher education. Graduate work 
in science is rarely graded on the basis of multiple-choice examinations; rather, such judgments rely on 
performance based assessments. Crucial decisions for academic tenure and promotion are based largely on the 
evaluation of a professor's portfolio; the evidence gathered by faculty members is evaluated against "criteria" which 
are used ty peers to judge faculty longitudinal productivity. Presentations at professional conferences or at 
research coiloquia constitute another example of academic performance assessment. Such presentations are 
virtually an integral part of the academic hiring process. Doctoral candidates defending their research are 
participating in a combination of portfolio and performance assessment when being judged by a dissertation 
committee of experts using what they believe is evidence of competence. 

The primary advantage of using the new and alternative methods of assessment are that they support 
broad, integrated curricula rather than discrete skills in decontextualized settings that are often artificial or 
contrived. Traditional tests are based on the idea of finding out what the student does not know. On the other 
hand, alternative assessments directly relate assessment results to learning objectives. The open-ended nature of 
portfolio-style assessments allows for the use of more authentic tasks that are analogous to real problems that real 
scientists might encounter (Shavelson, Baxter, & Pine, 1991). Portfolios further emphasize student creativity and 
responsibility through self-assessment and peer-assessment. 

The current literature concerning assessment strategies are deficient in providing guidance for 
improvements (Tobias and Raphael, 1995). Hestenes, Halloun, Ege, and Coppola (1992) found that students who 
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successfully pass introductory college science examinations frequently do not adequately comprehend Newton's 
laws of motion or the essence of structure and reactivity. Lin (1982) suggests that success in physics depends more 
on student test taking abilities than mastery of the concepts. In light of these observations, among many others, it 
is clear that educational research needs to develop and address alternative assessment strategies. 

Alternative assessment procedures posse s great possibilities in the introductory college physics 
classroom. Performance assessments can directly assess authentic scientific inquiry techniques such as 
measurement, classification, data organi 2 ation, communication of results, and inference (Slater & Ryan, 1993). 
Portfolio assessments provide students, and teachers, a mechanism to catalogue longitudinal progress and growth 
in scientific conceptions (Slater, 1994; Collins, 1992). 

Purpose of Study 

The purpose of this study was to address two major qae^ons with regard to the use of portfolios in the . 
content area of college physics: 

• Do physics students assessed by portfolios learn as much physics as their traditionally assessed 

coimterparts? 

• What are some of the affective consequences of using portfolio assessment procedures in an algebra- 

based, college physics course? 

These questions were explored by comparing two sections of an algebra-based, introductory college physics course. 
One group was assessed primarily by portfolio and the other by traditional essay-style exams. 

Method 

Particirrants 

The participants for this study were 35 freshman and sophomore undergraduate students erurolled in the 
first semester (15 weeks) of a two semester sequence of algebra-based, introductory college physics course at a 
large, urban community college in South Carolina. Students represented a variety of academic majors including, 
but not limited to, architecture, engineering graphics, computer aided design, pharmacy, pre-med, physical 
therapy, and education. 

Two sections of the course were offered; one section met during the day and the other in the evening. 
Students self-selected the section of the course in which they wanted to eruroll. The students had no indication 
prior to the first class meeting that one section would be assessed using an alternative assessment procedure. 

The evening section was selected at random to be the group using portfolio assessment as the primary 
student assessment procedure. The class was composed of 16 students (9 males and 7 females) and is referred to as 
Group P. The day class had traditional, student-supplied answer, essay-style tests as the primary student 
assessment procedure. The class was composed of 19 students (10 males and 9 females). It is called Group T. 

Research Design and Questions 

A two group comparison design was used in this study. Both groups were studied concurrently for one 
semester. Each group met with the same instructor twice a week for 75 minutes. The teacher-centered 
instructional strategy was designed to be as similar as possible. 

The project was designed to answer the following seven research questions related to student achievement 
and student perceptions of portfolio assessment: 

1) Did students in Group P demonstrate significantly different achievement on a common multiple-choice 

final examination than the students in Group T? 

2) Is there any significant difference in students' self-reported perceptions of achievement before and after 

instruction? 

3) Are the pre/post gains in student's self-reported perceptions of achievement different for students in 
Groups P and T? 

4) Is there a significant correlation between the student gain scores (posttest minus pretest) for students' 
perceptions of achievement and achievement on the course final examination? 

5) Is there a difference between Groups P and T in the correlations between gain scores on students' 
perceptions of achievement and achievement on the course final examination? 

6) How did the students in Group P feel about the use of portfolios as an assessment procedure? 

7) Did the students in Group P approach the learning processes in the course differently than usual? 
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Procedure 

The procedures used in this study will be described in detail in this section. The chronology of instruction 
and assessment for both groups can be found in Figure 1. Both sections of the physics course were taught diuing 
the fall semester of 1993. On the first day of class, prior to instruction, each group completed a survey of self- 
reported perceptions of achievement in physics. All participants were then given five weeks of instruction before 
being assessed by the instructor diuing week 5. Group P submitted their portfolios for the initial assessment at that 
time, while Group T took a 75 minute, traditional, short answer examination. 

The two groups were engaged in the second five weeks of instruction followed by assessment in week 10. 
At this point. Group P submitted their portfolios for the second time and Group T took an exam. The portfolios 
were graded and returned to the students. Following this, members of Group P participated in a focus group 
discussion concerning their perceptions of portfolios. 

Both groups received the final five weeks of instruction and were assessed a third time diuing week 15. 
After submitting their portfolios for the last time. Group P participated in a second focus group discussion 
concerning their perceptions of the effect portfolios have on their approach to learning. 



Insert Figure 1 about here 

During the last week of the study, both groups completed the same survey of perceived level of 
achievement which had been given the first ^y of class. This post-test was completed prior to taking an identical, 
four option, multiple-choice comprehensive final examinations. 

Instruction and Student Grading Procedure 

For all students, the course was designed to be taught in two discrete formats, a lecture 
portion and a laboratory portion. Only the lecture portion of the course was examined for this study. For group P, 
the portfolio grades counted 80% of the lecture portion of their grade and the final examination counted 20%. For 
Group T, the three in-class tests counted 80% of the lecture portion of their grade and the final examination 
counted 20%. 

Course Instruction 

Both sections received virtually the same classroom instruction by the first author during the same 
semester. The only intentional difference between the two sections was the method of assessment - portfolios or 
tests. 

The instructional strategy was primarily a teacher-centered discussion with emphasis on problem solving 
approaches. The instructor would first introduce a physical phenomena using demonstration apparatus, develop 
the mathematical formulas which describe it, and apply the equations in a variety of contexts. The instruction 
closely modeled the class assigned text written by Jones and Childers (1991). Students told by the instructor that 
they were free to ask questions at any time. 

Independent Variable: Level One - Portfolio Assessment 

Students in Group P submitted a portfolio three times (see Figure 1). The portfolios were evaluated to 
assess students' learning of the 24 course learning targets created by the first author. 

These learning targets were the same as the course objectives and are listed in Table 1. 



Insert Table 1 about here 



The three portfolio assessments were equally spaced throughout the semester with eight additional objectives added 
to the portfolio each time. 

Students were challenged to "document their adventure in learning physics." They were asked to create a 
personalized assessment portfolio that clearly demonstrated that they had mastered each of the specified learning 
objectives. The students were allowed to place anything they wished in their portfolio as evidence for mastering a 
learning objective. By way of further explanation, students were told that some recent portfolios had contained: a) 
essays concerning relevant news stories which demonjtrate a scientific concept; b) journal entries which describe 
how a topic is relevant/important; c) homework assignments suggested in class; d) short research papers; e) old 
tests from other classes; f) study guide quizzes; g) in depth analysis of a physical phenomena or common 
mechanical equipment; h) excerpts from a discussion amongst peers; i) relevant laboratory reports; j) reactions to 
the material presented in the te.\t; and k) creative homework problems. 
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All portfolio entries had to be clearly labeled and had to contain a statement of self-reflection. It was 
clearly emph^ized to the student that it was the student's responsibility to demonstrate to the instructor their 
mastery of the specified course learning objectives. 

Scoring rubric 

The scoring rubric for the portfolios, shown in Figure 2, was given to the students the first week of class. 
This physics portfolio scoring rubric was developed by the authors and modeled after one employed by Rischbeiter, 
Ryan, and Carpenter (1993). 

Each learning objective was judged to be a 0, 1, 2, or 3 based on the degree to which tlie evidence 
demonstrated competence with respect to the objective. Several portfolios were shared and scored among the 
authors who consistently agreed on the ratings. A numeric index of inter-rater agreement was not calculated. 
Students were allowed to question the portfolio scoring but there were no questions about any ratings given by the 
course instructor. 



Insert Figure 2 about here 



Independent Variable: Level Two - Traditional Assessment 

The students in Group T took three, 75 minute tests containing six multi-part problems and four short 
essay questions. The objettives/targets covered in each of the tests corresponded with those being addressed by 
Group P in the portfolios. The problems were written to be consistent with the problems given in the homework 
exercises and problems, 'fhe short essay questions were definitional in character. Scoring was done by the 
instructor using a traditional scoring scheme, deducting points for errors and omissions. 

Data Collection 

This section desaibes the data collection and analysis procedures for the study. The information is 
orgaiazed around the research questions. 

Quantitative Data Collection Instruments 

The quantitative portion of this study used a two-group comparison design to address research questions 
#1 through #5. The only intended independent variable was the method of assessment throughout the course, i.e., 
assessment portfolios or traditional examinations. The dependent variables used to answer the research questions 
were: 1) the final examination scores and 2) the gains achieved on a pre/post, Likert-style self-report survey. 
Dependent Variable One - Final Examination Score 

A comprehensive final examination was administered to both groups to address research question #1. The 
final examination was administered to both groups at the conclusion of the semester during a two hour final 
examination period. The exam was composed of 24 objective, four-option, multiple-choice items that corresponded 
to the 24 course learning targets (see Table 1). 

Approximately seventy percent of the examination items required some calculations. Few of the items 
were created to be strictly "knowledge level" items because the examination was administered in an open-book, 
open-note, open-portfolio format. In this context, a "knowledge level" item is testing the recall of facts. The test 
items were creat^ by the first author to be consistent in style and level of difficulty with the homew'ork problems 
assigned to the students from their textbook. 

Content validity of the exam was established by the use of expert judges. It was given to four, tenured 
college physics faculty who judged the 24 test items to be matched with the 24 learning targets established for the 
course. These judges also determined that the items were at the appropriate level of difficulty for the course. The 
Kuder-Richardson Formula 20 index of internal reliability (Gronland & Linn, 1985) was calculated to be 0.575 for 
both groups. This level of reliability is more reasonable than it might seem because the final examination was a 
comprehensive survey of the course with each item corresponding to a single, separate objective and, therefore, 
heterogeneous in nature. Measures of internal consistency for such heterogeneous tests tend to be lower than for 
tests containing homogeneous content material. 

Dependent Variable Two - Pre/Post Self-Report Survey 

A pre/post Likert-style self-report survey of student perceptions of achievement was to given to the 
students. The 24 course learning objectives/targets were used as the stimuli. This was done to for two reasons. 
First, such pretest formats appear to induce lower levels of anxiety in students. Second, the information gained 
from the surveys was needed to address research questions two through five. 




6 



Physics Portfolio Assessment 
page 6 

The students were asked to explain how well they imderstood each of the 24 learning targets (found in 
Table 1) by using the statement: 

"As of today, I have a blank understanding of each of the following items." 

They were to use the following number scale to fill in the blank: 

1 = very incomplete 2 = incomplete 3 = partially fragmented 

4 = good 5 = complete 

Cronbach's coefficient alpha index of internal reliability for both groups was determined to be .946 for the pre- 
survey and .929 for the post-survey. 

Qualitative Data Collection 

Group P participated in two focus group discussions according to the sequence previously shown in Figure 
1. They were both conducted by the second author. The initial focus group interview was conducted on November 
18, 1993 after the portfolios had been graded and returned a second time. Eleven students were present and all but 
one spoke. The interview lasted approximately 90 minutes. The second focus group interview occurred on 
December 9, 1993 and lasted approximately 35 minutes. Twelve people were present and eight spoke. Both 
interviews were held in the re^ar physics classrooms and were tape-recorded. Prior to each interview, the 
students were told that their names would not be included in the transcription. It was explained to them that this 
was done to insure their anonymity. 



Data Analysis 

Quantitative Data Analysis 

This section describes the analysis of the first five research questions which were analyzed using 
quantitative methods. P '•search question #1 concerned achievement as measured by a final examination. The final 
examinations were scored for the percentage of the 24 items correctly answered. The two groups were compared 
using independent t-tests. 

Research questions #2 and #3 concerned students' perceptions of achievement as measured by a pre/post 
self-report of knowledge survey. The raw scores were determined by summing the self-reported value (ranging 
from one to five) for the 24 items. The maximum value for the survey was 120 and the minimum value was 24. 
The raw gain score from the pre/post survey for each student was calculated (posttest minus pretest) and could 
result in a maximum value of 96. The posttest scores were compared to the pretest scores using dependent t-tests 
and the gain scores were compared between groups using independent t-tests. 

To address research questions #4 and #5, the correlations between the raw gain score on the self-report 
survey and the final examination score for the two groups were compared using the Fisher's Z-transformation of 
Pearson product correlation coefficients for two independent sample correlations (Freed, Ryan, & Hess, 1991). 
Qualitative Data Analysis 

Research questions #6 and #7 were addressed using the results from the two focus group interviews. 
Transcriptions of both interview were done by the first author after the course was completed. The transcriptions 
were verified by the second author. 

The moderator spoke 91 times in the first focus group and 41 in the second. In most instances these were 
single questions used to elicit clearer explanation of the topic being discussed. Students spoke a total of (194) 
times; their comments ranged from single utterances to discourses of a dozen sentences or more. After an initial 
reticence to speak, the students began very fast paced and spirited discussions of their opinions, perceptions, and 
positions on the subject of portfolios. 

Recurring themes were identified within the textual data gathered from the transcriptions. This was done 
in multiple stages: 1) the text was read repeatedly; 2) lists were made of themes that emerged; 3) the text was 
reread to check the themes for further division that might have emerged; and 4) the proceeding steps were repeated 
until no new themes were revealed. 



Results 

Quantitative Results 

This section describes the results of the final examination and the self-report of knowledge surveys to 
address research questions #1 through #5. The descriptive statistical information is presented in Table 2. 
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An independent t-test of means between the groups on the pretest sm'ey was calculated to determine the 
initial similarity of the groups. There was no statistically significant difference between the groups on the pretest 
(Mp=59.9, SD=18.9; Mj=61.1, SD=14.3; t(32)=.207, p=.837). This finding indicates that Groups P and T were 
similar at the beginning of the study. 

The final examination scores used to address research question #1 indicated that there was no statistical 
difference in mean scores of groups P and T (t(33) = 0.'' ’’2, g=.640). Group P had a mean score of 69% correct 
(raw Mp=16.6, SD=3.42) and Group T had a mean score of 71% correct (raw Mt=17.1, SD=2.72). 

Insert Table 2 about here 



Research question #2 explored the differences in perception of achievement from the begiiming of the 
course to the end of it. Students in both groups demonstrate significant gains on the pre/post survey as shovm in 
Table 2. 

Research question #3 looked at the differences in the self-report of achievement survey gain scores 
between the two groups. The scores indicated that there was no statistical difference between the groups (t(32) = - 
0.966, p=.341). The students in Group P did, however, have a slightly higher mean gain (Mp= 41.6, SD=20.3) 
than did their traditionally assessed counterparts in Group T (Mj= 35.6, SD=15.5). 

The descriptive statistics related to research questions #4 and #5 are listed in Table 3 



Insert Table 3 about here 



Research question #4 looked at the correlations between student gain scores and achievement measured by 
the final e.xam. The Pearson product correlation coefiicient for Group P was .493 (p=.052), whereas it was .526 
(p=.025) for Group T. 

For research question #5, a comparison of correlations between the gains on the pre/post survey to the 
final examination scores for Group P and group T shows that the correlation coefiScients are equivalent 
fe(33)=0.051). 



Qualitative Results 

Focus group discussions 

In general, the students had very positive reactions to the use of portfolios in their physics course. They 
seemed to like being assessed by portfolios instead of traditional exams. Four major themes emerged from the 
group discussion: 

• Many people don’t test well and using portfolios takes the pressure off. 

• Students approach their learning differently. 

• It makes a person work more consistently instead of “cramming” . 

• It reqmres. or allows, one to pay attention differently. 

The themes will be elaborated on separately in this section. There will also be a summary of the few negative 
responses to the use of portfolios for assessment. 

The results fi'om both focus groups are reported together. The second interview was held to help clarify 
the students comments about how they perceive their learning as a result of using portfolios. 

Theme One - Many people don’t test well and portfolio assessment takes the pressure off. 

Several students talked about how th^ don’t test well and that portfolios allow them to demonstrate that 
th^ ^ know the material. Th^ seemed to be grateful for an opportunity to show that th^ learned the physics 
without being subjected to the trauma of an exams. Individuals explained that portfolio assessment “takes away 
test shocks’ and reduces their anxiety. It was explained that this happens by allowing them to shift their focus from 
memorizing equations to demonstrating, in their own personal way, that they had met the course objectives. There 
was general agreement when one person said, 

“...most people who don 't test well, it 's not that they don 't know the information, you know, ...but [they] 
can ‘t show it on a test. " 
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Theme Two - Students approach their learning dififerentlv. 

Traditional physics courses, in the students* view, make them focus on writing down and memorizing 
equations. The portfolio approach, however, made them look at the broader picture of how physics applies to the 
^real world out there The following comment: 

‘7/ helps me learn things better and understand them, [better] than a test would'' 
met with agreement from several other students. 

There was discussion by the class on how they had to leant, not just memorize, the physics material and 
demonstrate that they knew it. They discussed how it caused them to have to take the time to understand the 
theory behind the problems they were working. The class talked about how portfolios require students to show the 
teacher what they have learned, and that they xmderstand the material and course objectives. Some people spoke 
positively of having to **show [the instructor] what you Ve learned," One individual articulated this sentiment by 
saying: 

*That 's what I liked about [portfolios]; because you are not memorizing it, and by writing it down [ to put 
into the portfolio] you will remember it longer. ” 

In both focus group interviews people commented on how well Dr. Slater taught. The question of teacher 
versus teaching method was brought up by one of the students who prefened taking tests to being assessed by 
portfolios. The point was made that Dr. Slater was a good, funny, up-beat teacher and maybe that was why people 
would remember things NOT because they had done the portfolios. Some people disagreed with this reasoning and 
stated that the portfolios were also integral in helping them remember physics. 

Theme Three - It makes a person work more consistently instead of “cramming”. 

The students, in general, thought that traditional classes allow them to procrastinate and wait until the last 
moment to “cram” in all the equatioiis they’ll need to take an exam. They talked about how this behavior causes 
them to forget everything shortly after the exam is finished. The portfolio, on the other hand, made them get more 
involved and woric more consistently on a regular basis. One person made this observation, 

'Tow can 7 put it off until the last hour. It is something that you have to do over a period of time" 

Another person stated, 

“/ Ve always done fairly good [on tests] but I like portfolios a lot better. And I *ve found that I did a lot 
more studying, research, whatever, on the portfolios than 1 would on a test. I was doing a lot more," 

This admission was voiced by others who said that they had done more studying and research in the course than 
they would have if they were only going to be assessed by exams. 

Theme Four - It requires, or allow, one to pav attention differently. 

The students said that they paid attention differently then in previous physics classes because thej^ were 
not worried about having to write down and memorize all the equations that were used. It was said that this 
allowed them to be more relaxed during the class period. Genei^ agreement was voiced when one student made 
the following remark: 

'7 think that., we pay attention differently. We don 7 just try to copy down the equations and memorize 
what this variable means... We, um, actually relate it to the big picture and we get to think about it, and 
think about what we are doing. " 

Many people commented on how they thought more about the application of the principles and the theory than 
mertiorizing formulas and doing problems. 

Another way in which the students indicated that they were pa 3 dng attention differently became apparent 
as they talked about the hands-on labs they had done. They mention^ that they could use the activities in their 
portfolios. When asked by the moderator whether they would have paid attention to a particular activity (miming 
up and down the staircase and calculating horsepower) if they were just going to be given a test on it, the replies 
generally reflected the thoughts of this individual, 

^If it was going to be on a test, you know, we go out there end do the activity. We come in, and. Oh God! 
We are going to have a test, and we \e been running up and down the stairs. The equation is such and 
such and such," 

There was discussion about how they were being more attentive and looking for examples of things to put into their 
portfolios in their everyday lives outside of the classroom. The portfolio made them think about physics by making 
them apply it to tlieir lives as reflected in this comment: 

"It just makes you pay attention [to things outside the classroom] so you can put things in a portfolio..." 
One event that took place during the course was the South Carolina state fair. It provided many students 
with opportunities to observe and describe the physics involved in the various rides and games. 
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Negative responses. 

Two students did not like having to do portfolios. Both stated that they do well on tests and believed that 
tests are a better reflection of a person’s knowledge. One made the following comment: 

**But to me, it *s busy work. I can team that information and take that test a whole lot faster than 1 can 
mess with that portfolio. ** 

Many comments which these two students made ran contrary to those of the other students. For instance, one said 
that tests ^ force you to study a little bit harder that just turning in the portfolio. This is opposite to what the other 
students felt. They believed that they studied more because of the portfolios. 

When comparing tests as a method of assessment with portfolios, one student said: 

“ I think that sometimes a person 's knowledge can Y be graded by a project like that [the portfolio].” 

The other made the remark: 

^^Ifyoti can do well on the portfolio [it] does not necessarily mean that you know the subject.'' 

The reverse of this sentiment was expr^sed by the others when they talked about portfolios compared to tests. 
Several people explained how they don’t test well and that the portfolio allowed them to show the instructor that 
they knew the material. They commented on how portfolios don’t create the type of anxiety that tests do. 

The two students who expressed negative views of the portfolio experience revealed by their comments 
that they were somewhat bewildered by what was expected of them. They were resistant to the requirement that 
they explain and justify what they included in their portfolios. When told that they could be creative in the ways 
they met the course objectives, they confused this with having to be ‘^humorous” and “funny”, and having to 
include a lot of pictures. These two individuals stated that they felt they could demonstrate their tmderstanding by 
simply including problems they had solved. One repeatedly mentioned that: 

I can work the problems but I have problems explaining to him, teaching him, you know... ” 

The second person said, 

“...Ae wants the personal, your personal, you know, insight - how you see physics. But I just don Y see 
why he should force you to do that. If you understand the material you should be able to get the grade 
out, if that's the wav vou want to do [it]." 

It appears that these individuals are comfortable with the traditional method of assessment and find portfolio 
assessment diJBflcult and frustrating. 

One individual eloquently summed up the two alternate ways the people in the course preferred to be 

assessed: 

“ It *s not the portfolios; it *s not the tests. It 's just individual people. Some people do not do good on 
tests.. ..Others don Y like portfolios 'cause maybe, maybe they just don Y like to spend all the time involved 
to do one; to do a good job to prove what they have done. They would rather just take the test, get it over 
with, and be done with it. " 



Summary aivd Discussion 
Measured Achievement 

A direct comparison of final examination scores revealed that there was no statistically significant 
difiefence between Groups P and T. Students who have been assessed using portfolio-style assessments 
demonstrate equivalent performance on a traditional, multiple-choice final examination as did students assessed 
using the traditional method. 

Tliese results indicate that students who are creating portfolios do not learn less than their traditionally 
assessed counterparts. Indeed, it is quite possible that the students who created portfolios experienced a greater 
increase in comprehension of physics which might not have been revealed on a traditional, multiple-choice test. 

Self-Reoort of Achievement 

A comparison of gains in students* self-report of achievement in physics revealed that both groups report 
enhanced knowledge of physics concepts after instruction. Group P did have a slightly higher affective gain than 
did Gi'oup T but the difference was not statistically significant. Students in Group P believed that they had learned 
the material even though they were not motivated by impending traditional tests. 

Correlation Between Achievement and Affect 

An examination of the correlations between achievement and self-reported gains of knowledge revealed 
that there were no statistically significant differences between the two groups. Group P demonstrated a slightly 
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lower correlation between achievement and gains than their traditionally assessed counterparts. This observation 
might be the result of portfolio creating students having the perception that they have achieved high levels of 
physics concept comprehension. Alternatively, it is also possible that students might have achieved higher levels 
of comprehension that are not adequately measured using a traditional, multiple-choice final examination. 

Furthermore, the relatively high correlations between achievement and gains on the self-report of 
knowledge survey suggest that such a survey is reliable. Such an instrument is valuable because it is easy to use, 
efficient to score, and does not appear to induce anxiety on the participants. 

Attitude Towards Portfolio Assessment 

The students in Group P reported that they liked this alternative procedure for assessment. Perhaps most 
importantly to the students, the portfolios were reported to reduced their levels of "test anxiety." Furthermore, 
evidence from the focus groups show that these students thought that they had a better understanding of physics 
because the portfolio was being used as the primary assessment tool in the class. 

In Ae past, students have reported that they "freeze up" or "forget everything" when they sit down to take 
a traditional, achievement test. It is recognized that this could be real anxiety, a learned anxiety firom other 
students, or an excuse for poor performance depending cn each individual student. Regardless, the portfolio style 
of assessment seems to reduce tlus perceived testing anxiety. This reduction in student anxiety clearly shows up in 
the way that students attend to class discussions. The students report that they are not nearly as worried atout 
permanently remembering what each variable represents because they will look that up in the text later as 
necessarj\ 

Students suggested that they feel they are being relieved of the traditional rigorous note taking duties to 
“capture the formulas.” This allowed them to look at the holistic physics of a given situation - not just the 
formulas. They stated that they enjoy class discussion more because of the atmosphere promoted by the assessment 
strategies employed. 



Students Retxmed Approach to Learning 

Students assessed by portfolios reported that they spend more time going over the textbook or required 
readings to make sure that Aey comprehend the depths of each learning objective than they would have if they 
were being assessed traditionally. Although it is unclear exactly how much time students devote to creating their 
portfolios, they do report that they contemplate physics outside of the classroom environment - always looking for 
that "neat thing" to include in their portfolio. 

Certainly, the amount of time that a student devotes to his or her studies varies considerably among 
students. However, the creation of a portfoli'j does appear to require sustained effort on the part of the student. 
Furthermore, the learner must reflect on the salient points of a class lecture/discussion. This process encourages 
the student to grasp the holistic and integrated view of a concept. 

Diuing class discussions, students were trying to figure out how to apply current in-class discussion in 
their portfolio. They accomplished this by actively comparing the discussion with their outside of class 
experiences. During the focus group discussions, students reported that in many traditionally assessed classes, they 
would take rigorous notes, memorize them for the test, and then forget everything. Participants in the portfolio 
asse^ment concluded that they thought they would remember the material they were learning in physics much 
better and longer than material learned in other classes they took. They believed that they had internalized the 
material while working with it, thought about the principles, and applied physics creatively and extensively over 
the duration of the course. 



Instructor Observations 

The course instructor observed that the instructional environment was slightly different by the end of the 
course for students in Group P as compared to Group T. During class time, students in Group P appeared to ask 
many more questions than students in Group T. Members of Group P, in addition to common questions about 
solving textb<x)k-style physics problems, seemed to ask more questions that involved real-world applications of 
physics. This changed the instructional environment somewhat in that such questions often lead the constructivist 
instructor to describe more complex and 'interesting* phenomena. Clearly, this particular affect on the 
instructional environment needs to be observed more closely in future studies. 

It was not possible to conduct this project as a purely experimental study due to the logistics of this 
specific college environment. In particular, there are acknowledged uncontrolled variables present in this study 
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such as the self-selection of students, the 'personality* of the class, the class meeting time, and previous experience 
in science course work. The researchers, however, were cognizant of these and other issues throughout this study. 

Conclusions 

An analysis of the results of this study strongly suggest that portfolio assessment procedures are as 
effective as traditional assessment approrches at enhancing conceptual understanding and attitudes toward 
learning and assessment in the college physics classroom. Moreover, the processes used to create an assessment 
portfolio do not seem to detract from traditional examination performance. The use of portfolios appears to have 
the additional benefits of lowering test anxiety, encomage students to use physics in non-textbook situations, 
allowing for longitudinal observations of achievement, and promoting student responsibility and introspection. 

Portfolio assessment procedures allow instructors to view student achievement in a longitudinal and 
holistic perspective. The college physics classroom learning environment is positively enhanced when students 
embrace the task of creating portfolios as evidence of their abilities and comprehension. The creation of a portfolio 
requires the learner to go beyond the traditional expectation of concept recognition in the classroom. This process 
encourages students to devote considerable time to their studies. Moreover, when students draw connections to 
physics concepts beyond the classroom, as many do in preparation of portfolio entries, their knowledge becomes 
more broad and less discretely packaged. The learner is encoiuaged to seek out physics concepts in the physical 
world and describe these experiences as evidence of learning. As instructors, it is this higher level of achievement 
that instructors most desire their students to achieve. 

Students often complain that physics courses are irrelevant and boring - something apart from the real 
world. Portfolios seem to provide a motivation and a mechanism for learners to organize their knowledge in their 
own meaningful terms. Students manipulate examples taken from their real world experiences for portfolio 
entries. Students do more than work assigned problems; they earn ownership in their learning. Portfolio-style 
assessments appear to have the distinct advantage of helping students to take responsibility for learning physics. 
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Table 1 

Course Learning Targets Used for First Semester Alecbra-based College Physics 



1 . Understand the nature of scientific knowledge and the various disciplines of science. 

2. Appreciate the historical and practical uses of units and measures. 

3. Convert numerical quantities from one system of units to another and within a given system. 

4. Describe the various concepts and units used to describe motion. 

5. Solve one dimensional problems related to the acceleration of objects due to gravity. 

6. Diagram and describe quantitatively the motion of a projectile. 

7. Appropriately apply vectors qualitatively to describe physical situations. 

8. Use vectors to quantitatively solve problems relating to motion. 

9. Create a ffee-body diagram to represent the total force on an object (including friction). 

10. State and apply the laws of motion developed by Isaac Newton. 

1 1 . Solve problems related to static equilibrium and rotational equilibrium. 

12. Apply the Law of Universal Gravitation to objects moving in circles. 

1 3 . Calculate the work done on an object and its relationship to energy. 

14. Quantitatively and qualitatively describe systems in which energy is conserved. 

15. Identify the various sources of energy and power. 

16. Solve problems related to impulse and the Conservation of Momentum. 

17. Apply principles of fluid dynamics to describe phenomena in nature. 

18. Distinguish between heat and temperature. 

19. Identify the ways that heat can be transferred between two points. 

20. Explain the distinguishing characteristics of solids, liquids, and gases. 

2 1 . State the laws of thermodynamics and their importance to technology. 

22. Solve problems relating to periodic (cyclical) motion. 

23. Describe the properties of sound waves with respect to pitch, volume, and intensity. 

24. Apply the Doppler Effect to physical situations quantitatively and qualitatively. 
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Figure 2 

Portfolio Scoring Rubric 
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Portfolio Assessment Scoring Criteria: Each item in the student portfolio should 
clearly indicate which of the eight learning objectives it demonstrates. The individual 
evidence (each piece) is scored as follows; 

0 = No evidence: the information is not in the materials identified. 

1 = Weak evidence: the information is presented bu t unclear, partially 

incorrect, or implies or reflects misunderstandings . 

2 = Adequate evidence: the information is presented with no errors nor 

misunderstandings implied but information is dealt with at the 
literal descriptive level with no integration across concepts. 

3 = Strong evidence : ^he information presented with no errors nor 

misunderstanding implied in a comprehensive and integrated 
fashion. 

Portfolio Evaluation Scheme; The overall portfolio is scored with respect to 24 
objectives as follows as an indication of the extent to which the portfolio indicates that 
the student has mastered the course objectives. : 



Grade. 
A : 
B+: 

B : 
C+: 

C : 
D+: 

D : 

F ; 



Rubric: 

Strong evidence in at least 20 objectives; adequate in the other four; 
Strong evidence in at least 20 objectives; adequate in at least two others; 
Strong evidence in 18 objectives; adequate in the other six; 

Strong evidence in 16 objectives; adequate in the other eight; 

Strong evidence in 16 objectives; adequate in at least four others; 
Adequate evidence in all 24 objectives; 

Adequate evidence in at least 18 objectives; 

Adequate evidence in less than 18 objectives; 
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Table 2 

Mean Scores (Raw Scores') Compared by Group 



Portfolio 

Group 

(Np=16) 



Traditional 

Group 

(Nt=19) 





Mp 


SD 


Mt 


SD 


t(df) 


Significance 
Between Groups 


Number Correct on Exam 


16.6 


3.42 


17.1 


2.72 


0.472(33) 


NS 


[percent correct] 


[69%] 




[71%] 








Pretest Survey 


59.9 


18.9 


61.1 


14.3 


0.207(32) 


NS 


Posttest Survey 


101.4 


9.38 


95.4 


10.5 


-1.789(33) 


NS 


Self-Report Gain 


41.6 


20.3 


35.6 


15.6 


-0.966(32) 


NS 



Note; Both instnunents had 24 items. Self-Report Gain (maximum value of 96) is Posttest Survey 
minus Pretest Survey for each student. NS indicates not significant at alpha = .05 level 
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Table 3 

Pearson Product Correlations Compared by Group 



Correlation Coefficients Between Final and Surveys (E-value) 









Self-Report Survey 








Pretest 


Posttest 


Gain 


Group P 

Final Exam Score 


(Np=16) 


-.356(.176) 


.351(.183) 


.493(.052) 


Group T 

Final Exam Score 


(Nt=19) 


-.372(.128) 


.479(.038) 


.526C025) 



Note; The correlations between Survey Gains and Exam Score for both groups are statistically 
equivalent at the a = 0.00 1 level (Fisher’s z = .05 1). 
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